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This contribution aims at studying the behaviour of the classical sample moment estimator, 
S{n,q) = Y^k=i'^k/^' ^ ^ function of the number of available samples n, in the case 
where the random variables X are positive, have finite moments at all orders and are 
naturally of the form X = exp Y with the tail of Y behaving like e"^*^ . This class of 
laws encompasses and generalizes the classical example of the log- normal law. This form 
is motivated by a number of applications stemming from modern statistical physics or 
multifractal analysis. Borrowing heuristic and analytical results from the analysis of 
the Random Energy Model in statistical physics, a critical moment qdn) is defined as 
the largest statistical order q up to which the sample mean estimator S{n. q) correctly 
accounts for the ensemble average EX*, for a given n. A practical estimator for the critical 
moment qdn) is then proposed. Its statistical performance are studied analytically and 
illustrated numerically in the case of i.i.d. samples. A simple modification is proposed to 
expHcitly account for correlation amongst the observed samples. Estimation performance 
are then carefully evaluated by means of Monte-Carlo simulations in the practical case of 
correlated time series. 
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1. Introduction 

Motivation. Estimating moments of a given order q > from a finite size observation 
of a given times series {Xf., k = 1, . . . , n} appears as both a natural and simple problem. 
In the general and common case where little a priori is known nor assumed on the data, 
the classical sample moment estimator for the order q, 



is natural to use (positive random variables X, with finite moments of all orders only are 
considered, for reasons made explicit below). Probability theory provides practitioners 
with valuable results regarding the performance of the sample mean estimator, notably in 
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the asymptotic limit of an infinite observation duration, n — >■ +C!0 (cf. e.g., [5]): The weak 
law of large numbers shows that the estimator is consistent; The central limit theorem 
further precisely quantifies the asymptotic limit. However, such results can also be read 
as relevant solutions to a rather abstract situation where one is interested in a particular 
statistical order q and may have at disposal an observation of potentially infinite length. 
In practice, the natural situation can often be formulated in a converse way: Practitioners 
often work hard to obtain an observation of length n and can naturally wonder how 
large is the order of moments that can actually be estimated correctly given the observed 
{Xk,k — 1, . . . This dialectic opposition between the theoretical results and the 
practical questions is sketched in Fig. 1. 

Even for the case of interest here where all moments are finite, EX'^ < +oo, it is 
well-known that the correct estimation of moments of all orders is impossible, from any 
finite size observation. Indeed, for a fixed n, 

S(n,q) - - max Xl^ElX'^]. 

This behavior that can be referred to as a linearization effect, as for fixed n, the practical 
plot, log 9 vs. log S{n,q) systematically appears as a straight line, in the limit q — >■ +oo. 
Therefore, a natural question stems: Given an observation of finite size n,up to which 
order moments can actually be correctly estimated ? When X belongs to a class of, say, 
simple random variables (defined by the fact that their characteristic function is analytic 
in 0), and for i.i.d. observations, a bound is given in [16]: 

Inn 

1c[n) oc . (1) 

2 in in n 

Though a valuable result, this remains of limited practical use as it is restricted to 
i.i.d. observations belonging to a simple class, and, first and foremost, consists of an 
asymptotical result with no explicit prefactor. 

Surprisingly, this question seems to have received little research efforts. In common 
real world data analysis, this is however a question of obvious interest. This is notably the 
case in scale invariance analysis, increasingly used in many applications of very different 
natures (cf. e.g., [1] and references therein for a review), where scaling is measured 
via the dependence with respect to the analysis scale a of the moments of order q of 
multircsolution quantities computed from the observation {Xk, k = 1, . . . , n}. Such an 
issue also appears, though in a less obvious way, in the physics of disordered systems, 
a representative example of which being the so-called Random Energy Model (REM) 
[9, 19], where the partition function 

Z(n,/3)=^exp(/3n), (2) 

k 

(a fundamental quantity in the statistical physics framework) needs to be evaluated, with 
/3 — l/ksT, T the temperature of the system and ks the Boltzmann constant. The 
variables are (the opposite of) random energies, drawn from a Gaussian distribution 
in the original version of the model [9]. Such a choice leads to a log- normal statistics for 
the variables X^ = exp(/?Yfc). The REM has however been extended to other random 
variables, such as log-exponential-power laws, obtained by choosing a distribution of Yfc 
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Figure 1: {'n,q) plane and linearization effect. Left, theoretically, the performance of the sample 
moment estimator S{n,q) are characterized, for a given q, in the limit n — > +00, i.e., moving along a 
horizontal line in the (n, q) plane. Practically, however, the central limit issue consists of determining how 
large is the order q of the moments that can actually be correctly estimated, from a given observation of 
finite size n, i.e., moving along a vertical line in the (n, q) plane. Right, practically, in the limit q — > +00, 
for a fixed n, lnS'(n, q) necessarily behaves as a linear function of q, this is illustrated here for a log-normal 
distribution with three different values of n ( n = 10'^ (red circle), n = 10^ (green square), n = 10® (blue 
triangle) ; the solid black lines show the theoretical In EX'' ). It shows that both the slope of the linear 
behavior and the departure from In EX'' depend on n. This contribution aims at defining and estimating 
practically a critical order qc{n) up to which the sample moment estimator correctly accounts for the 
ensemble mean EX''. 



with a tail p{y) ~ exp{—cy^), with 6 > 1 and c > 0, when y — > +00 [7]. This similarity 
between the REM and the moment estimator issue raised above stems from the close 
resemblance between the partition function Z(n, {3) and the sample moment estimator 
S(n,q) for random variables X}^ = expY^, with Yj, belonging to the simple class. 

Another interesting example stems from multifractal analysis. Indeed, popular mul- 
tifractal processes are defined from the celebrated Mandelbrot multiplicative cascades 
(cf. e.g., [18, 11]), where the multipliers entering the construction of the cascade are also 
chosen a.s X — exp(— F). 

Goals, Contributions and Outline. In this context, the present contribution first 
aims at defining a critical order qc{n) up to which moments of order q can be estimated 
from a given finite size observation of length n, belonging to the class of positive RV 
X = expF, where the class of RV to which Y belongs is precisely defined in Section 2. 
The definition of the critical order qc{n) is based on a detailed heuristic argumentation 
borrowed from the analysis of the REM [9, 19] and is also strongly grounded on the 
thorough theoretical asymptotic analysis conducted in [3]. This construction is detailed 
in Section 3. Second, a procedure for the practical estimation of qdn) from a finite 
duration observation is devised and its performances are studied theoretically in the 
case of i.i.d. observations (cf. Section 4). Third, the estimation procedure is slightly 
modified to account for potentially dependent observations and its performances are 
thoroughly studied by means of Monte-Carlo simulations (cf . Section 5) . The Matlab 
practical procedure implementing estimation will be made available publicly at the time 
of publication. 
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2. Class of random variables 



Log-normal. As mentioned in Section 1 above, the typical example of interest here 
is that of log-normal random variables X. In that case, Y — InX simply consists of a 
normal random variable, whose cumulative distribution Fy satisfies ln(l — Fy^y)) ~ 2/^/2 
for y — >■< +00. In the present contribution, following extensions naturally encountered in 
both the REM and Multifractal contexts and the theoretical framework envisaged in [3], 
a much larger class of random variables is considered by extending the log-normal case 
in two ways: The power law behavior y'^ is extended to y'^. The power-law behaviour is 
defined through asymptotic properties which capture the essential characteristics of exact 
power- law. 

log-exponential-power law. The class of log-exponential-power law random variables 
X is defined as follows: 

X^expY, (3) 

where the cumulative distribution function of Y, Fy is assumed to be a continuous 
function that satisfies: 

l-i^y(2/) = e-''to). (4) 
Furthermore, the function h is required to verify: 

h{y)^L{y)yP, with p > 1, (5) 

where L is a slowly varying function (cf. e.g., [4]), i.e., a function satisfying 

Vt, hm — — = 1. 
1/-S-+00 L[y) 

In addition, for technical reasons made clearer in Section 3, this class is slightly restricted 
to twice-difl^erentiable functions L satisfying: 

liiii = 0, (6) 

lini fy^y = 0. (7) 

These condition impose a certain form of regularity on the derivative of L. Notably 
Eq.(6) excludes slowly varying function with increasingly fast oscillations. Similarly Eq.(7) 
constraints the behaviour of L" with respect to L and L' . 

Consequences. From the definition above, a number of properties for the random 
variables Y and hence X can be derived. The probability density function py of Y reads: 

py{y) = h'{y)e-'^^y\ (8) 

All moments of X arc finite and can be written as: 

/ + 00 /• + 00 

h'{y)e'^y-''^y'^dy= / h'{y)e'^y~y''^^yUy. (9) 
-OO J —oo 
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Nevertheless, the moment generating function of X, 



r+ca 

E[e'^]= h'iy)e'^''P'-y^-^^y^y"dy, (10) 

Jo 

is finite on R~ only and its characteristic function is hence not analytical in (see [17] 
for more detailed consequences). 

Interpretation. Eq.(5) together with Conditions (6) and (7) essentially control the 
behavior of the right tail of the distribution of Y and hence define a broad class of 
random variables X, whose moments are all finite, but with characteristic function not 
analytical at zero. The interest of that class of random variables hence stems from its 
lying between the simple class of random variables, whose moments are all finite and 
whose tail decreases exponentially fast (whose characteristic function is hence analytic at 
zero, e.g., the absolute value of normal random variables) and the so-called heavy tailed 
class, whose moments remain finite only up to a given order (such as e.g., the Pareto 
distribution). The parameter p plays a key role in studying that evolution from simple to 
heavy tail: while, strictly speaking, both classes are excluded from the class studied here, 
they can be approached by letting p — > +oo and p — > 1, respectively. 

Local power law exponent. In Definition (5), the parameter p is naturally referred 
to as the power-law exponent. However, in the course of the study reported below, a local 
power-law exponent, defined as follows, will play a crucial role, where local refers to a 
pre-asymptotic behavior, i.e., for finite y: 

din h{y) yh'{y) 

It is straightforward to verify that Conditions (6) and (7) imply: 

lim pi{y) = p, (12) 

1/-5- + 00 

lim p[iy) = 0. (13) 

1/-5- + 00 

Examples. In the numerical analyses conducted in Sections 4 and 5 below, two explicit 
examples of p-parametrized families of such random variables X are used. 

• The log-WeibuU distribution defined as: 

F(?/) = 1 - e"^^ with p > l,y > 0. (14) 
This notably implies that the local power exponent is constant 

V2/,Pi(y) = P- 

• The strict log-exponential-power law distribution defined as: 
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where F denotes the gamma function [21]. This implies that its probability density 
function reads: 

2r(l + i) ^ ' 

It can be shown that using the asymptotic properties of the incomplete 7 functions 
that 

lim pi{y) = p. 

V i-+oo 

• The log-normal distribution corresponds to a special case of the strict log-exponential- 
power law family, with p = 2. The following relations can be derived: 

h{y) = ln2-lnerfc(^), 
v2 

^'^^^ V2^erfc(y/v^) ■ 

The asymptotic form of the complementary error function erfc, in the limit y — > +00 
: erfc(?/) ~ ^ yields: 

h{y) ~ ln2+ |- + ln(y2^2/), 
lim pi(y) = 2. 



3. Critical order: combining dominant contribution to finite size efftects and 
truncated moments. 

This section aims at unveiling a change in the behaviour of the sample mean moment 
estimator, 

1 " 
fc=i 

in the (g, n) plane, for a critical curve qdn), that hence depends on the observation sample 
size n. This is obtained by combining a a moment dominant contribution argument to a 
truncated moment finite size effect analysis. This analysis is strongly inspired by those 
commonly conducted in statistical physics, notably for the study of the Random Energy 
Model [9] and closely resembles a glass transition. 

3.1. Moment dominant contribution 

The behavior of the moment of order g of X can first be related to the tail of its 
distribution and hence to h(y). Eq.(9) can be rewritten as: 

^0 
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Using a saddle-point technique [8], in the limit q +00, enables one to evaluate the 
integral above as (technical proof detailed in Appendix A): 



where y* is determined by 

q~h'{y*) + ^{y*)^0- (20) 

Qualitatively, y* denotes the value of y that essentially contributes to the moment 
of order q, E[X'']. Eq. (20) implicitly defines a relation y*{q), that depends only on 
the precise details of the distribution of Y. Because Condition (7) imposed in Section 2 
precisely aims at constraining its amplitude, the term h" /h' can be read as a correction, 
depending of the fine local behavior of py- Indeed, under this condition, one can show 
that: 

y-5-+oo h'(y) ' 

which hence implies that: 

h'iy*) « q. (21) 

3.2. Finite size effect: truncated moments and sample moment estimator 

Let us consider a finite size observation of n i.i.d. observations {Yfe, k £ 1 . . . n}. Its 
samples lie between empirical minimum and maximum = min{Yfc} and Mk = max{ Yfe}. 
A finite size observation thus explores only a bounded range of values in the existence 
domain of y. When n — ^ +00, the bounds of this range naturally extend to cover the 
entire domain of definition of Fy. However, for finite n, this bounded range is likely to 
affect the properties of the sample mean estimator for a given q, if the information needed 
to evaluate this moment (qualitatively y*) lies beyond the observed M^. This section 
aims at detailing and quantifying this statement as well as at drawing its consequence in 
terms of sample moment estimator properties. 

3.2.1. Finite size accessible range 

Let us first define a frontier beyond which the probability to observe a sample = y 
within a finite size observation is low. Let t denote an arbitrary positive constant, that 
controls the probability that maxlYfe, k — 1, . . . , n} exceeds a frontier y\: 

Vroh{yk=l,...,n,Yk<y\) =e-\ 

Using the independence of the {Yfc, fc € 1 . . . n} and the relation above yields, 1 — F{y\) = 
- + o(-), or equivalently, h{y\) = Inr + Inn + o(l). For the class of random variables 

considered here (cf. Eq.(5)), for n — > +00, one has: yX ^ (l + jj^) y\. This indicates 
that it is natural to simply consider j/^(n), defined as: 

h{y\n))=\mi. (22) 

which quantifies the upper bound of the tail of y actually likely to be observed from an 
observation of finite size n. Let us note that y^{n) depends both on n and on the details 
of the tail of the distribution of Y. 
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3.2.2. Truncated moments 

Essentially, the finite size analysis above states that, for a given observation of finite 
size n, the domain of existence of F, ]R, must be split into two distinct subranges: 

- (— oo,j/^^), a reachable range which contain most observed samples,^ 

- [y^ , +00), an unreachable range, within which the probability of observed samples 
is very low. 

This leads us to define the so-called truncated moments: 

MT{n,q)^ e9a-/i(y)+inMy)rfy. (23) 



^0 

This truncated moment provides an heuristic estimate of the typical value of the moments, 
where the contributions of atypical term is removed. 

3.3. Sample moment estimator and truncated moment 

The sample moment estimator q) = X]r=i -^i ^'^^ observation of n i.i.d. 

random variables Xi = e^', in the class of power law exponential, of interest here, can 
be related to the truncated moments as follows. Let q{n) denote a function of n that 
satisfies: 

3e > 0,g(n) < (lnn)2-i/p-^^ (24) 

then, as n — )• -|-cxd. 

In S{n,q{n)) ^.^^ In MT{n,q{n)) ^^5) 
In n ?i-s-+oo In n 

Essentially, this means that, asymptotically {n +00), on condition that q does not grow 
too fast (cf. Condition (24)), the (In of the) sample moment estimator converges to the 
(In of the) truncated moment. The complete proof of this results is detailed in Appendix 
B. In essence, it relies on the specific properties of the class of random variables studied 
here and closely exploits in spirit the analogy with the REM, extending reasonings such 
as those in [20]. 

3.4. Phase transition and critical moment 
3.4.1. Critical frontier 

Combining the moment dominant contribution evaluation (cf. Section 3.1) to finite 
size truncation effects (cf. Section 3.2) yields two distinct situations depending on the 
relative positions of y*{q) and y\n), and hence separated by: 

yHn)=y*{q,). (26) 

This equality implicitly defines a frontier that splits the {q, n) plane, introduced in Sec- 
tion 1, into two zones corresponding to different behaviours of the truncated moments (in 



^For positive moments, the positives value of Y play a dominant role, so one can focus of the positive 
frontier Howrever, a natural extension to negative moments would introduce the negative counterpart 
2/1 of which would play an exactly symmetric role. 



the REM framework, this is a phase transition) . This frontier can either be formulated as 
a critical order qdn) or as a critical size nc{q). 



For y*{q) < {n) (equivalently for q < qc{n)), the saddle-point argument enables, as 
detailed in Appendix A, the evaluation of Mt as 



which by comparison to Eq. (19), is hence shown to correspond to InE [X']. By nature, 
the saddle-point argument implies q — > -l-oo, which combined to q < qdn) yields that the 
result above is valid for q — >■ -|-c!0, n — > -l-oo, with q < qdn). 

Conversely, when y*{q) > y^(n) (equivalently for q > qdn)), the dominant contribution 
to MT{n, q) is no longer controlled by y*{q) but instead by the border of the integration 
domain y^n). Therefore, the saddle-point argument now yields, in the limits q — > +00, 
n — > +00, with q > qdn) : 



This clearly shows that In Mt depends linearly on q, for large q. 

3.4-2. Sample moment estimator 

The almost sure convergence established in Eq. (25), and proven to be valid regardless 
of the relative values of y*{q) and j/^(n), indicates that the phase transition is also 
transferred to the behaviour of the sample moment estimator, which reads, in the limits 
q — >■ -|-oo, n — >■ -|-oo, with qM{n) — {lnn)'^~^/P~'^ satisfying Condition (24): 



As can be seen comparing Eqs. (24) and (34) below, for e € (0,1) qdn) «C quin). 
Therefore, for a large range of gs, the sample moment estimator S{n, q) correctly estimates 
the ensemble average EX' for q < qdn), while liiS{n,q) behaves linearly in q when 
q > qdn)- 

This phase transition is illustrated in Fig. 2 from numerical simulations. A large 
number of independent realizations of n i. i. d. log-normal RV X enable to compute an 
average value of In S{n, q), for various n and q. This average can further be compared 
to InE [AT*]. This clearly validates that for q < qdn). In S{n, q) corresponds to InE [A''], 
while In S{n,q) becomes significantly lower than lnE[A'^] for q > qdn) and behaves 
linearly in q. Results obtained for various n can be superimposed by plotting In S{n, q) 
and InE [A'] as a function of q/qdn) rather than as a fimction of q, where qdn) is 
obtained by solving numerically Eq. (26) for each n. 

3.4.3. Critical moment 

Combining Eq. (26) with Eqs. (20) and (22) yields: 



liiMT{n,q) = qydq) - Hydq)) + In /^'(y* (g)) , 



(27) 



In MT{n,q) = gy^(7i) - /i(y^(n)) + In /i'(j/^(n)). 



(28) 




(29) 



^ ^ 1 y^h'jy^) 
Inn j/t /i(yt) 



h{yf)h'{y^) 




(30) 
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Figure 2: Quantitative predictions for the linearization effect. Left, ElnS'(n, g) as a function of 
q, witii predicted values of qc, for tlirce different values of n; n = 10^ (red circle), ji = 10^ ( green square), 
n = 10^ (blue triangle) ; the solid black lines show the theoretical InEX^. Right, E [lnS'(n, q)]/E [Xi] 
as a function of q/qc, for the same 3 different values of n. Eln5(n,g) is obtained from Monte-Carlo 
simulations, with a Log-Normai distribution. These plots clearly validate the relevance of the defined 
critical moments qc{n). 



Given Conditions (6) and (7), the second term on the right side of the equation above 
can be shown to be neghgible compared to the first one. Moreover, on the right-hand-side 
of Eq. (30), the local tail exponent pi can be recognized: 



Further defining, 
qdn) can be approximated as: 



^ p^ivHn)). (31) 



^H-^, (32) 



,(n)~pz(yt(n))0(n), (33) 



which will prove of particular interest for the practical estimation of qdn). 

As an interesting corollary, the growth rate of Qc as a function of n and p can be 
evaluated. Eq. (5) implies that there exists a slowly varying function £, closely related 
to L, cf. [6], such that: 

h-\liin) = C{liin){liiny/P, 
Substituting this into Eq. (33) leads to 

qc{n) = pC{liin){liinY^p . (34) 

When p — > +00, this is reminiscent of Eq. (1), recalled in Section 1 and valid for the 
simple class of random variables. However, when p decreases and approaches 1, this 
critical moment Qc increases far more slowly with n. As an example, for the log-normal 
case, p = 2 implies that qdn) (x Vln n. 
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4. Estimation 



This section aims at defining an estimation procedure for qdn) that can actually and 
practically be used from a finite sample size observation {Xi, . . . Xn}. Also, its statistical 
performance are studied both theoretically and numerically. 

In Section 3 above, it has been shown (cf. Eq. (33)) that qdn) can naturally be split 
into two components 9{n) and pi{y^{n)). An estimation procedure is hence devised and 
studied independently for each components in Sections 4.1 and 4.2, respectively, yielding 
an estimate of Qc as a product of the estimates (cf. Section 4.3). 

4.I. Estimation of 9 
4.I.I. Definition 

A straightforward calculation not reported here shows that y^{n) = h~^{lnn) = 
Fy^il — ^), which indicates that the definition of 9 (cf. Eq. 32) can be rewritten in 
terms of the (1 — l/n)-th quantile of the distribution of Y: 

0-^^- (35) 

The estimation of 9 can, therefore, essentially be recast into that of (the inverse of) the 
(1 - i)-th quantile of F. 

Classically, the p-th quantile is estimated, from an observation of sample size n, using 
the order statistics of rank [p.n] [5]. In the present context, this amounts to using the 
largest value of the observation, 

My,„^ Max{Yi,...,r„}, (36) 

as an estimate of the (1 — -)-th quantile. For the class of random variables Y considered 
here, My^n belongs to the domain of attraction of Gumbel law [10, 13, 12]. This implies 
that there exists a sequence a„ > such that [A/y.„ — Fy^(l — ;^)]/a„ converge in 
distribution towards a Gumbel distribution with a cumulative function F(x) — exp{—e^x). 
Consequently, 



7, (37) 



where 7 is the Euler-Mascheroni constant [21]. This straightforwardly indicates that My^n 
constitutes a biased estimator for Fy'^{\ — ^) for finite sample size and that it is not, in 
general, asymptotically consistent. 

To overcome this drawback, the estimation procedure can be refined by involving 
not only My_„, but the fc-largest values in the observation {Yi, . . . ,Yn}. Therefore, let 
{Yin,i = l,...,n} denote the ordered list (in descending order) of the observation 
{Yfe, k = 1, . . . , n} and let ke(n) be a function of n satisfying (for reasons made clear 
later) : 

lim M!^ = 0. (38) 

n— i-+oo n 

A parametrized collection of estimators of 9 is defined as (with k = kg{n)): 

9i^^ - (39) 
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where flk consist of Hnear combinations of the order statistics: 

k k 

flk ^ wit^^ ^ = 1. (40) 

i=l 1=1 

Obviously, the case fc = 1 amounts to using Afy_„ only. 

4-.1.2. Performance: theoretical analysis 

To study the performance of 9'^''\ it is needed to analyze the statistical properties of 
flk, which can be rewritten as 

where 



In the limit n — > +oo, the {Ui, i = 1, . . . ,k = kg{n)} converge in distribution (— >■) toward 
a random vector: 

(C/i,...,C/fe)4AWEE(Gi,...Gfc), 
whose joint distribution reads [10, 12]: 

PA^ (ffi, • ■ • gk) = lffi> ->sfc exp ^-e-S" - ^ ft j . (41) 

The change of variable A^*^) = {Gi,...Gk} ^ A'^*"^ = {{A^ = i{G^ - G^+i), i = 1, . . . , /c - 
l},Gfc} yields: 



PA/J(5i,...4-i,5fc) =exp ( -e ~ kg^ ~ "^sA , (42) 

\ i<k ) 

which shows that asymptotically (n — > +oo), the {Ai,i = l,...,fc— 1} arc independent. 

From these definitions, a number of properties of fifc, for fc > 2, have been derived 
and are presented below. 

Proposition 1. On condition that it is applied to a random vector that exactly follows 
PAfc; the estimator fife that has minimal variance under the constraint that it is unbiased 
for finite n (i.e., EQk = -^^^(1 — ^) or, equivalently , E^^^j^ ctiGi = ) is obtained by 
setting {ai, i — 1 . . . ,k} to: 

\/i,i^k a, = ^ |l'=^ ' (43^ 

Q!fe = 1 - (A; - l)ai 
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This stems directly from the following results (whose proofs are postponed to Appendix 
C.l) 



E[Gk] 


= 7 - 


fc-1 

^ • 


(44) 


Var [Gk] 


6 


fc-1 ^ 


(45) 


E[A,] 
Var [A,] 


= 1, 
= 1. 




(46) 
(47) 



Proposition 2. When n +00, k +00 with ;| —> 0, ilfc is consistent. 

This is a direct consequence of the combination of Proposition 1 above with the fact 
that for all random variables in the class studied in this contribution, the random vector 
{Ui, i = 1, . . . , fc} converges in distribution towards A'^'^^. Proof is further detailed in 
Appendix C.2. 

Proposition 3. When n — > +00, k — > +00 with ;^ — > 0, fifc is asymptotically normal. 
When applied to a random vector that exactly follows , can be split into 

i 

Using the fact that the vector {A^, i = l,...,fc — 1} consists of unit variance i.i.d. random 
variables yields 

rife + E [Gfc] - Gfc V , , ^ 1 

(C(l;fc-l)-7)^ m, 1) where ((.; = E ^- 



and 

This implies: 



E[Gfc]-Gfe4 



AA(0,1). 



(C(l; fc-1) -7)2 

This shows that fife is normal for finite n when applied to a random vector that exactly 
follows pAfc • Asymptotic normality in general follows from the convergence in distribu- 
tion of {C/i, i = 1, . . . , fc = fce(n)} towards A'^'^^. Proof is further detailed in Appendix C.2. 

The consistency of 6'^'^'^ immediately follows from that of fi^. However, it should be 
noted that 0*^*^^ is not asymptotically normal. 

These analytical results are obtained with choices ko{n) satisfying the constraint in 
Eq. (38). Choosing precisely kg{n) is however an intricate question that can be formulated 
into a classical bias- variance trade-off one: Large fc should yield low variances while small 
fc are needed to ensure convergence toward a law of extremes (A*-*^^) and hence a low bias. 
This trade-off is now studied by means of Monte-Carlo simulations. 
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Figure 3: d(^s) as a function of kg: relative biases and MSEs. Left: Log-WeibuU distribution; 
Right: strict log-exponcntial-power law. From top to bottom: p = 1.2,2,4, n = 1000. Dashed lines: 
relative biases; Solid line: relative MSEs. These plot essentially shows the benefits of increasing kg from 
1 to 2. They also indicate that kg can be further increased with benefits as long as kg <g n. This is true 
for all p. 
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Figure 4: 0^^^^ as a function of p: relative biases and MSEs. Left: Log-Weibull distribution; 
Right: strict log-exponential-power law. Top: relative MSEs; bottom: relative biases, n = 1000. Red 'o': 
fee = 1 ; green '□': kg = 2 ; blue 'A': kg = 25 ; cyan '0': kg = 50. These plots show again the benefit of 
increasing kg from 1 to 2, and there exists an optimal kg that depends essentially on n and weakly on p. 



4-. 1.3. Performance: numerical analysis 

Numerical simulations are conducted using both categories of examples, log-WeibuU 
and strict log-exponential-power law distributions, described in Section 2, for different 
values of p, and different observation size n. For all simulations, 500 independent 
realizations of n i.i.d. random variables are drawn. 

From these, the (relative) biases, variances and mean square errors (MSE) of 9^^'^ are 
evaluated and reported in Fig. 3. For both types of laws, it shows that, as expected from 
the theoretical analysis above, increasing fcg from 1 to 2 yields a significant decrease in 
bias, and hence in MSE. For fcg = 2, ai ~ Q!2 ~ \, hence showing that a simple arithmetic 
mean of the first and second largest observed values performs much better than the use 
of the sole maximum My^n- The bias only slightly further decreases when kg is increased 
beyond 2, before it finally increases again when kg becomes non negligible compared to n. 
Combining biases and variances, the MSEs show, for all p, the benefits of using kg > 2, 
yet small, kg <C n, compared to kg — 1 or to kg too large (too close to n). Also, it can be 
noted that the closer p to 1, the larger kg needs to be to achieve the lowest MSEs. 

Furthermore, Fig. 4 reports the relative MSEs (left) and biases (right) as a function 
of p, for various choices kg. Besides confirming that the benefit from using kg = 2, or 
slightly larger, against kg — 1 is valid for all p, it also shows that biases and MSEs are 
roughly constant for p >2, while, in most cases, significantly increasing when p — >■ 1"*". 

To finish with. Fig. 5 compares kg^opt (i-e., kg that yields the lowest MSE) to tentative 
candidate functions for practical kg{n): kg{n) ~ 10 Inn — 40, kg{n) = exp(\/l.61nn). 
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Figure 5: d^'°^^: kQ ^pt and MSEs as a function of Inn. Left: Log-WeibuU distribution; Right: 
strict log-exponential-power law. Top: kg ^ptt bottom: MSE. Red 'o': kg = exp(-v/l.6 In n) ; green '□': 
kg, opt opt = min of MSE. p = 2. These plots show the relevance of the proposed empirical formulae for 
ke,opt{n). 
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Though empirical, those formula hold, for both types of distributions, for a large range of 
values of p (p g [1.5, 3]) and of n, and can hence be used as rules of thumb for a practical 
use of the estimator. 



4-2. Estimation of pi{y\n)) 
4-2.1. Definition 

For simplicity, pi{y^{n)) is hereafter relabeled as ps. From its definition (cf. Eq. (31)), 
Pe can be interpreted as the local exponent governing the behavior of the function h 
(cf. Eq. (4)) around the largest available observation y^{n). This suggests the direct use 
of the empirical cumulative distribution function to estimate pe. Indeed, for a sufficiently 
small interval around y\ one has d\nyh{y^) = pe, which enables us to write: 

ln(- In Fy (y^)) «PB In (48) 

Considering the ordered list the (1 — :^)"th quantile can be approximated by Yi^n, 
hence: 

In (^-In^^ w In K,,„ + /3. 

This suggests that pE can be estimated by a non weighted least square fit that involves 
Yi,n for i < kp, where kp is sufficiently small compared to n: 

C(lny,ln(lnn-lni)) 
^ CilnyMy) ' ^''^ 

with X = I X^i'^i ^^'^ ^(^1 y) = xy - xy. 

4.2.2. Performance: theoretical analysis 

Again, the choice of kp constitutes the critical issue that can be expressed in terms of 
the usual bias-variance trade-off: Because pE is a local exponent, that may converge only 
slowly to p, it needs to be estimated on narrow intervals around y^, hence the use of a 
small kp; Conversely, large kp decrease the variance of the estimate at the price of a bias 
increase. Actually, a too large kp necessarily leads to the underestimation oi pE, as can 
be seen by evaluating the variations of pi{y) for each Fi_„ and Yk^n- This motivates the 
choice of kp according to: 

lim = 0. (50) 

)i— S-+00 n 

4.2.3. Performance: numerical analysis 

The performance of pE as well as the trade-off framing the choice of kp{n) are studied 
numerically by means of Monte-Carlo simulations conducted as described in Section 4.1.3. 

Relative biases and MSEs are reported in Fig. 6, as a function of kp. It shows that 
for the log-WeibuU distributions, bias is negligible and the larger kp, the lower the MSEs. 
This stems from the fact that PEin) = p, Vn, which implies that pE is unbiased whatever 
kp, and hence that increasing kp decreases the variance at no cost. However, this is a 
particular case that departs from the general situation better accounted for by the second 
example consisting of strict log-exponential-power law distributions. On this second 
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Figure 6: P^'' as a function of kp-. relative biases and MSEs. Left: Log-WeibuU distribution; 
Right: strict log-exponential-power. From top to bottom : p = 1.2,2,4, n = 1000. Dashed blue lines: 
relative biases ; Solid red lines: relative MSEs. For the Log-WeibuU distribution, because Pe(p^ = Pi 
increasing kp is beneficial as it only yields a reduction in variance. However, in general, there is an 
optimal kp^opt that depends on n and p. 
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Figure 7: p^'' as a function of p: relative biases and MSEs. Left; Log-WeibuU distribution; 
Right: strict log-exponential power. Red 'o'; fcp = 30 ; green fcp = 50 ; blue 'A': kp = 100 ; cyan '0': 
kn = 200. 
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Figure 8; p^'' : kp^opt and MSEs as a function of Inn. Left: Log-WeibuU distribution; Right: 
strict log-exponential power. Top: kp,opt, bottom: MSE. p = 2. Red 'o' : kp{n) = 87i'^ * ; green '□': 
kp,opt opt = min MSE for k < 1000. p = 2. These plots show the relevance of the proposed empirical 
formulae for fcp,opt(n). 
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example, Fig. 6 indicates that the optimal choice of fcp(n) depends on the actual value of 
p and must be chosen larger for p — > 1. 

Relative biases and MSEs are reported in Fig. 7, as a function of p. It shows that there 
is a uniform in p benefit to increase kp for the log-WeibuU distributions. For the strict 
log-exponential-power law distributions, however, kp^opt is larger when p — >■ 1. 

To finish with. Fig. 8 compares fcp,opt (i-e., k that yields the lowest MSB) to tentative 
candidate functions for practical kp{n): kp(n) = 50 Inn, kp{n) — 8n^^^. Though empirical, 
those formula can be used as rules of thumb for a practical use of the estimator. 

4-. 3. Estimation of 
4-. 3.1. Definition 

From ^C^f) and p^'''' defined above, an estimator for Qc can be proposed as: 

qike,k,) ^§ike)~ik,)^ (51) 

4.3.2. Performance: theoretical study 

To study the performance of qc, its bias B\qc\ = — 9c and variance Var [q^ can be 
computed as 

B%] = Cov[^, pe] + B[e]{pE + B[pe]) + B[pE]{e + B[e]), (52) 

and 

Var [gj = Var 9 Var[p£;] 

+ CoY[e\pl]~CoY[e,pE? 

+ Var [^1 {pE + B[pe]? + Var [pe] {9 + B[9]f - 2Cov[9, pe]{pe + B[pe]){9 + B[9]) 



Furthermore, assuming that the relative biases are negligible yields: 
Bm B[9] , B[pe] , Gov[9,pe] 



Pe 



PeO 



(53) 



Var [qc 



Var 9 Ya.r [pe] 



{peOY 

Cov[02,p|;]-Cov[0,p£]2 



Var 



612 



{PEe? 

, Var[p£;] ^Cqy[9,pe\ 



P\ 



PeQ 



These calculations show that the performance of qc significantly depend on the covariance 
between the estimates 9 and p. This covariance is investigated by means of numerical 
simulations. 
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Figure 9: Correlation between §(''e,opt) and p^'' "''*'. Left: Log- WeibuU distribution ; right: Strict 
log-exponential-powcr distribution. Top: as a function of p, n = 1000. Bottom: as a function of n, p = 2. 



4-.3.3. Performance: numerical analysis 

Montc-Carlo simulations, in the spirit of those presented in previous sections, with 
the empirically found best choices for kg{n) and kpin), are conducted. Fig. 9 shows that 
Cav[9,pE\ remains quite high and positive, whatever p and n, and hence significantly 
contributes to the bias and variance of <fc- Furthermore, results reported in Fig. 10 
essentially show that the MSEs for vary from 5% to 25% depending on the actual 
distribution of Y and the value of p, with larger MSEs for p — > 1. Though these MSEs 
may seem high, still provides practitioners with a practical procedure yielding a very 
satisfactory order of magnitude of the true q^. In applications, one is rarely interested 
in the precise value of qc, while the knowledge of a estimated order of magnitude often 
proves very important as soon as data interpretation is concerned. 

5. Correlated time series 

So far, observation {Xi, . . . ,Xn} consisting of independent samples only have been 
dealt with, when we are mostly interested in stationary time series analysis, where data 
are likely to be dependent. The goals of the present section is therefore to analyze the 
impact of dependence amongst observations on the definition and estimation of qc as 
well as to propose a simple and elementary modification of qc as described above to 
accommodate dependencies. 

5.1. Theoretical analysis 

In Section 3 above, the critical moment arose from a balance between two competing 
factors: dominant moment contribution versus finite size effect. While the definition and 

21 



0.5 
0.4 
0.3 
0.2 
0.1 
0.0 



1 




























































. H 


- 


1 





1 


1 



1.5 2.0 2.5 3.0 3.5 4.0 
P 





10^ lO'' 10^ 
n 



Figure 10: qc' relative biases (blue dashed line) and MSES(red solid line). Left: Log-WeibuU 
distribution ; right: Strict log-exponential power distribution. Top: as a function of p, n = 1000. Bottom: 
as a function of n, p = 2. n = 1000 



evaluation of the former depend only on the marginal distribution of the data and are 
not affected by dependencies, the latter is far more impacted by correlations. Indeed, a 
finite size accessible range has been defined by 

Prob (Vfc = 1, . . . , n, Yfe <y^)^^, 

For independent data, the previous equation simplifies to h{y^) — Inn. For correlated 
time series, it is natural to expect that y\, can be defined via an effective number of 
sample rCp, as: 

h{ylp) — In riy. 

From this modification, the analysis conducted in Section 3 leads to define a critical 
moment under dependencies as: 

ql{n) = qc{n*T). (54) 

As for the independent case, can be split into two terms, 

q^{n)^pl{n)0^{n). (55) 

Reinjecting yJp, into the definition of and 6'^ leads to 

plin) = pE{n*T), 0^{n) = 9{n*r). (56) 
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Figure 11; '' : Impact of dependencies. Applied to correlated observations, (jc is impaired by 

a significant structural bias that cannot be accounted for by a rescaling (of the form suggested by Eq. 

(58)). (kffjkp) = (1,100). Black line: true Qc, gc*" ''''', for r = 10 (red circles), r = 50 (green squares) 
T = 100 (blue triangles). 



Let T denote the correlation length of the analyzed time series, defined as r = 
J^°°tC{t)dt/ J^°° C{t)dt. When T is well defined, it is natural to assume that n* es- 
sentially behaves as n* oc with the obvious requirement that limT-_>.o n*(n, r) = n, 
suggesting the phenomenological form: 

(57) 



1 + kt' 

with K a positive constant to be determined. This leads to: 

qJ{n)^q,{T,n)^q,{n/{l + KT)). (58) 

5. 2. Impact of dependencies on qc 

To study the impact of dependence on qc, the estimator is applied to a large number of 
independent realizations of stationary time series, of length n, and with a priori and jointly 
prescribed (log-Weibul or strict log-exponential power law) marginal distributions and 
covariance function C{t). These are synthesized numerically using the Hermite expansion 
and Circulant Matrice Embedding based techniques described in [15, 14]. To understand 
the impact of dependencies, let us analyze the case where C {t) takes the simple form: 

C(t) = e-^, (59) 

with T a tunable constant, obviously corresponding to the correlation length. 

Results, reported in Fig. 11, show first a strong discrepancy between Qc and qc{T,n) 
and second and mostly that no rescaling operation of the form suggested by Eq. (58) 
can account for it. Indeed, a rescaling as in Eq. (57) amounts only to a translation of qc 
along the horizontal axis. Moreover, assuming n* < n restricts to left translations only, 
which makes it impossible to superimpose all qc{T,n) curves by rescaling: qc does not 
converge towards qc(T,n). 

To further analyse the cause for this discrepancy, let us analyse separately ^('^s) and 
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Fig. 12 obtained from the same Monte-Carlo simulations, with kg = 1, indicates that 
a k(« 0.08) can be found such that E « ^h!n* (l^ite logically 6 = lnn*/r2 is to be 

used instead of 9 = Inn/fl, as In n in the definition of 9 does not scale with t.) This means 
that fii exhibits the expected behaviour, E [ili{n, t)] = E [rii(n*)]. However, increasing 
kg from 1 to 2 and above does not, as in the independent case, brings a reduction in 
bias, but instead significantly increases it. This can easily be understood as very likely 
the second largest value is located extremely close to the first largest one and highly 
correlated to it, hence not bringing any bias reduction. Instead, this correlation prevents 
the convergence of the order statistics to its asymptotic limit and hence impairs the 
consistency of fi. 

This significant correlation of the order statistics also explains the poor performance 
of pe- As illustrated in Fig. 13, pe does not converge towards /3_e(t, n) and no rescaling 
can account for this discrepancy. This can be interpreted as follows: pE strongly relies on 
the convergence of the fcth order statistics towards the (1 — fc/n)-quantile. Correlations of 
the sample generates correlations amongst this order statistics because correlated samples 
tends to regroups together in clusters in terms of ranks. So the correlation of the samples 
is transferred to the order statistics, which dramatically hinders the convergence of the 
order statistics towards the quantile. 




Figure 12: 9: Impact of dependencies. Rescaling of as a function of n* = n/(l + kt), for 
T £ {10,50, 100} (red circles t = 10, green squares r = 50 , blue triangles r = 100). The black line 
correspond to the theoretical value 9{n) / Inn. 

Therefore, applied to correlated stationary times series, (jc, as defined in Section 4, 
cannot converge towards qj. However, a simple modification described below is shown to 
significantly account for correlations: It consists of imposing that the largest value has 
been found, the second largest value used must be picked far enough from the first largest 
one, the third largest must be picked far far enough from the two first ones, and so on. 
The modified hence essentially consists of the original qc, applied to these far enough 
order statistics. 

5.3. Modified estimators 

An efficient way of implementing the idea described above is to construct the set of 
effective independent samples step-by-step. Consider the original sample Vi = Xi, . . . , X„. 



24 



6 7 8 9 10 11 12 
Inn 



Figure 13: pE' Impact of dependencies. Blaclc line: theoretical value, red circles r = 10, green 
squares r = 50 , blue triangles t = 100. 

First, the maximum, Xk-^ — maxj^Pi} is selected. Then, all samples located close to Xk-^ 
in time arc eliminated: using the distance 

dp{Xi,Xj) = ma.x{\j — i\, /3Ca.rd{k,mm{Xi, Xj) < Xk < max{Xj, Xi)}). 

the set Vi is sieved by eliminating samples belonging to the sphere centered at and 
of radius s. A new set V2 = ViX Bdp (X^-^ , s) is defined and the procedure is repeated 
until a set Ve = {^kn ••■,^fei} of effective independent points is obtained. The modified 
O'^ and ff^ are defined by applying the original ^ or pE to Ve- 

Monte-Carlo simulations conducted as above suggest that the threshold should take 
the following form: 

s — ar, (60) 

with the choice a w 0.01, as illustrated in Fig. 14. 

These same Monte- Carlo simulations also show that the estimation of 9 once corrected 
(by assuming the value n* is known) leads to the expected 0{n*) (cf. Fig. 15) and that Qc 
can thus be very satisfactorily estimated (cf. Fig. 16). 

This very good performance however involved the a priori knowledge of r, and 
Monte-Carlo simulations provided accurate estimates of the parameters a and k. These 
parameters would obviously not be known on actual data and would need to be estimated. 
Such estimations are beyond the scope of the present contribution. They could be based 
on spectrum estimation or other classical techniques. This is under current investigation 
and will be the subject of another contribution. However, to conclude this study. Fig. 17 
reports the expectations and standard deviations of , applied to log- normal exponentially 
correlated data with r = 100, when the value of r actually used on is, in purpose, 
incorrectly specified. Satisfactorily, Fig. 17 shows a weak sensitivity of with the value 
of T actually used, notably when it is larger than the actual correlation length. This means 
that, in practice, a rough estimation of the true correlation length might be sufficient to 
obtain satisfactory performance of . 
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Figure 14: Modified estimator p|J. fcp = 100. Black line: theoretical value, red circles t = 10, green 
squares r = 50 , blue triangles t = 100. 
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Figure 15: Estimation of 9{n*) with the corrected estimator 9'^ . kg = 10. Black line : theoretical 
value, red circles r = 10, green squares r = 50, blue triangles t = 100. 




4 5 6 7 8 9 10 11 12 
Inn* 



Figure 16: Estimation of qc{n*) with corrected estimator. Black line: theoretical value, red circles 
r = 10, green squares r = 50, blue triangles t = 100. 
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Figure 17: q^: Influence of the choice of r. Solid black lines show the average value and standard 
deviations of using various values of r. The vertical dashed line corresponds to the actual correlation 
length of the data, r = 100. The horizontal dashed line indicates the theoretical value qc{n,T). 

6. Conclusions 

In the present contribution, the question of defining and estimating a critical order qc 
up to which the moments can be correctly estimated from a sample observation of given 
size has been addressed. More specifically, the case where the observations stem from 
distributions that have finite moments of all orders yet significantly heavier tails than 
Gaussian random variables (such as e.g., the log- normal distribution) has been considered 
as it arises in many modern applications, such as e.g., the Random Energy Model in 
statistical physics or Multiplicative Cascades in Hydrodynamic Turbulence. 

It has been shown that this critical order can be given a formal quantitative definition, 
combining dominant contributions and finite size effects. This definition clearly highlights 
the impact of the parameter p quantifying the heavytailness of the distribution on the 
growth of the critical order qc as a function of the size of the sample (the more standard 
case of variables having tail of the order of that of Gaussian random variables or a 
power-law tail being retrieved by taking the limit p — >■ +oo and p — >■ 1). Monte-Carlo 
simulations validates the practical relevance of this quantitative prediction. An estimator 
that can actually be applied to a single realization of data has been defined, and analyzed 
both theoretically and by means of Monte Carlo simulations, in the context of i.i.d. 
observations. This procedure provides practitioners with a tool that can practically be 
applied to a single observation of a given actual data. Matlab procedures implementing 
this estimation procedure are available upon request. 

The question of dependence amongst observations has then been addressed and shown 
to significantly decrease estimation performance. However, a simple modification of the 
estimation procedure has been devised to account for dependence. Its effectiveness is 
validated by Monte-Carlo simulations. The use of this procedure on actual data only 
requires a rough estimation of a typical correlation length amongst observations, which 
can be achieved using classical signal processing tools (such as spectrum estimation). 

The definition and assessment of an original procedure systematically integrating the 
estimation of the typical correlation length within that of qc is under current investigation 
and will be the subject of a future contribution. 

An analysis, elaborated along the same line, has recently been proposed to define 
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the critical moment for another class of variables: that corresponding to the increments 
of multifractal processes, that differ (significantly) from the class considered here in 
having finite moments only over a finite range of orders as well as long range correlations. 
The reader is referred to [2] for comparisons enabling insightful understanding of the 
mechanisms at work in defining a critical order. 
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Appendix A. Saddle-point evaluation of EX'' 

The saddle-point method consists in a classical technique enabling the evaluation of 
the asymptotic behaviour of integrals of the form g{x)e*-^^^''dx for large t [21, 8]. 
More precisely, when x* is the only maximum of /, the saddle-point evaluation reads : 



This technique is used here to evaluate the asymptotic behaviour of E [X"^] when 
q — > +O0. Eq. (18) leads to define the logarithmic density ip{y) = h{y) + lnh'{y). The 




(A.l) 
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maximum of the integrand in Eq. (18) is hence located at y* , such that ip'{y*) = q. The 
change of variable y = vy* enables to rewrite Eq. (18) as: 

p+ca 

E[X'^]=y* e'^y'^-'^'^'^y'^di^. (A.2) 

Moreover, Conditions 6 and 7 imply that 'ipiv) ^ h{y) and ip'{y) ^u-y+oo h'{y). 
Combined together, these results yield: 

qy*t ~ i;{ty*) = i^{y*){pv ~ + e{y\v)), 

with \/v, limj,._j.+oo e(2/*: v) — 0; and hence 

p + CO 

Jo 

The above equation can also be rewritten as 

f+oo 

E[X'^] — y* e'''^*^^^^'^ / g'/'(y*)(i-p+P''-»^''+s2(j/*,i'))^j^^ 

with £2(1/*, v) — e{y* , v) — e(y*, 1). For v ^ 1, 1 — p + pu — + £2(2/*, f^)) < thus 
only the neighbourhood oi v — \ contributes significantly to the integral when y — > +00. 
Moreover, for v close to 1 

y*ilj'{y*) f * w *2'^"{y*)v'^ , ( 2-s 

Injecting this approximation into the integral above leads to a Gaussian integral 



Jo 

and hence to Eq.(19) : 



/ 2tt 



E[X«] - e«^*^'^(^*\/— (A.4) 

Appendix B. Truncated moments and sample moment estimator 

In this section, the almost sure convergence of f{n,q) — In S{n, q)/ Inn towards 
lini„ In Mxin, q)/lnn is proven. 

The most fundamental point is to make precise the role play by yl as a frontier. By 
definition, 

Prob(yi,...,y„ >2/t) = i_e-^ ~ T 

The Borcl-Cantclli lemma [5] states that if a sequence of random events An satisfies 
^P(A„) < +00, then the events only occurs a finite number of times. 
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Here, a natural choice of events would be A,-, = {3i < n, Y, > y\n). If rip) < +00 
then for sufficiently large n, all the Yi are almost surely smaller than However, to 
preserve the property yl ^ y\, it is also required that In r <C In n. Unfortunately, these 
two conditions arc incompatible. 

To answer this dilemma, let us consider a subsequence n(r) of samples (for instance 
n(r) = 2'' ) and events = {3i < n{r), Y^ > yln{r)). Then, the condition in the 
Borel-Cantelli lemma becomes : 

^T(n(r)) < +00 (B.l) 

r 

One should note that this condition, with n(r) = 2'", is compatible with In r ^ Inn if 
one choose r = 1/ In n. Moreover, if the value of In S/ In n at points n(r) is known, the 
behaviour of / can be inferred for all points. Considering two points k < I, one has: 

(0)1^ + 1-^ 

With a = n{r) < k <b = n{r + 1), the inequality below holds: 

, ,,lna _ Ina ,,,, ,,,,ln6 , ln6 z,.^ „n 

/(a,a(a))j^ + 1 - li^ < /(^'^(^)) < fM'))]^a + ^ " h^" (^-'^ 

With ln(n(r + l))/ln(r) 1, the above inequality describes precisely /, in the limit 
n — >■ +00. This equivalently amounts to defining n(r) as 

n(r) = exp(e'^^'"'), with lim i/(r + 1) — z^(r) = 

r— ^+00 

A simple solution would be to choose v{r) — ^/r and T(r) = ^, then n(r) = exp(e^) 
and 

r(n) = ^ 



In Inn 



Therefore, if points n(r) only are observed, for r sufficiently large, almost surely, all the 
Yi are smaller than j/J. 

Denoting xi the characteristic function of the set /, 

, , fl ifa;e/ , , 

X/(a;)= . ^, . , (B.3) 
otherwise 



one can write 



This sum can be further evaluated by spliting the remaining interval (—00, yl] in 2R{n) + 1 
sub-intervals with 

ak = k- 



R{n) 

h = (afe,afe+i],fce -R,. . . ,R-1. 
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I-R{n)-1 = I^oo = (-00, a_fl] 

For reasons made explicit later, we choose 

„ / s In n ^ 
R{n) = -— . B.4 
In In n 

Because £(lnn)(lnn)^/'', the length of the sub-intervals is converging towards 0. 

S{n, q) can hence be rewritten as 

n R(n)-1 
i=l k=-B.(n)-l 

It is possible count the number of point in each interval: 

n 

Under certain conditions Nk should not differ too much from 

ENk - nExi, (B.5) 
More precisely, Markov inequality [5] provides us with an almost sure upper bound: 

2i?(n) + 1 



Prob(VA:, A^fe > aENk) < 



To use Borell-Cantelli lemma, it is required that J2r '^^a{nlr))'^ ^ +00. The slower a 
grows, the more information can be obtained from Borel-Cantelli lemma. A valid choice 
here implies In a <C In 71. Due to the sparsity of n(r), 

a{n) ^ R{n)lnn, (B.6) 

is a compatible choice. This yields: 

R{n)-1 

Sin,q)< «WE [X/J 6'"'-+^ (B.7) 

fc=-_R„-l 

Furthermore, let ip denote the logarithmic density (as defined in Appendix A), 

E[xi,]<2{y^/R)eM-^M). 

The major contribution to the sum comes from the interval containing y* (defined as 
ip'iij*) = q). This leads to a rough upper bound, 

5(n, q) < a{n) (yt e^a^+^-v-Ca^) ^ ^ [xi_J e"'^') . (B.8) 

Most of the terms in this equation are logarithmically negligible compared to In n. 
Therefore, 

lini ^"^-^('^'g) < lini Qarn+i{n) ~ iP{am{n)) ^ 
n In n n In n 
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The lower bound is more difficult to obtain and the method proposed here uses the 
Bienayme-Tchebitchev inequality and is thus only applicable to i.i.d. variables. 



Prob(iV.>M.)<^ (B.IO) 



2R 

= (lnn)(2''"^"^/'')(^+'(^)) 

This latest equation imposes the most restrictive condition on R. Indeed, this implies 
that 

EProb(A',„>'yf!-)<+oo^ 

The Borel-Cantelli Lemma can be used to obtain 

1 9a,„-^(a„+i) (B.ll) 

Um ^"-~/(^'"+^) <lim/Kg). (B.12) 

n In n n 

The last needed step is to verify that the two bounds converge when n — > +00. It is 
needed to verify first that 

i?lnn (Inn) 2 
The convergence comes at the price of a condition on q 

3e > 0,g< (lnr^)2-l/''^^ 

The second term is verified easily 

^'(y^)y^ In Inn 
i? In n In n 

Thus, the image of /,„ hy y qy — "ipiy) converges towards a simple point. 
Combining these results with (B.12) and (B.9) leads to 

^.^^ InSM a^s ^.^^ qy*{n) -^iy*) ^ 
n In n n In n 

which hence, combined to a saddle point evaluation of the truncated moments proves 
Eq. (25). 
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Appendix C. Estimation of 6: theoretical results 

Appendix C.l. Construction 

In order to prove the consistence and the asymptotic normality of Ofe, it is easier to 
consider variables (Gi, . . . , G„) of law A''^). If we then define 

Ai = i{Gi-Gi+i), (C.l) 

one can see that 



PA(fc)((5i, . . .,Sk-i,gk) = exp I -e - fcgfc - i ■ Aj j . 



3 



(C.2) 



In other words, Aj and are independent. Moreover, if we call 

n 

Some computations then leads to : 



E[Ai]=l, (C.4) 

Var[Ai] = l. (C.5) 
Similarly with 7 the Euler-Mascheroni constant: 

E[Gfc]=7-C(l;fc-l), (C.6) 

Var[Gfe] = '^-C(2;fc-l). (C.7) 
If we now try to apply fifc to the Gj variable, we can translate 

nk = Y,aiGi (C.8) 

into 

k-l 



^^fc = Gfc + ^AAi. (C.9) 

i=l 

with Pi = i 12j<i ^j- We can thus calculate the quadratic standard error with 

E [nl] =E[n]^+ Var [n] . (C.IO) 

which brings forth 



^[^fe] = ^-C(2,fc-i) + 5jA'+ (7-C(i;fc-i) + |jA] . (C.ii) 



j=i \ 1 
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We can then minimize E [^1] directly according to the f3i with a Lagrange multiplier 
A in order to insure than £[17^] — 

k-l 

aft(E [nl] - AE pk]) = 2(7 - C(l; k-l))-X + 2Y,(3j+ 2A- (C.12) 
An obvious solution is then Pi — fi— '''•^'^^1"'^''' a-^d so : 

ak = l- {k~ l)ai 

Appendix C. 2. Consistence 

We can now verify that limfc E [fi^] = for the variables Gk 

E [nl] = ^ - c(2, fc - 1) + (Cii;k-^i)^-i)\ (C.14) 

The result comes immediately from the fact that C(2) — Using the convergence in 
distributions of the Uk, we then obtain 



E[n^{U)]^0. (C.15) 

k 



Appendix C.3. Asymptotic Normality 
One can write f2/j as 



fe-i 



f^fe = 3fe + /3^A,:. (C.16) 



From Eq. C.2, it is easy to verify that the variables are i.i.d.. Thus 

f^fc-.gfc-C(i;fc-i) + 7 V 



C(l;fc-l)-7 

Moreover, 



AA(0,1). (C.17) 



E[g,-C(l;fc-l) + 7]=0, (C.18) 

Var [gk ~ C(l; A: - 1) + 7] = - C(2; k - 1), (C.19) 

6 

implies that 

(7fe-C(l;fc-l) + 7 4 0. (C.20) 
It is thus possible to eliminate this term from the previous equation: 

Q,k V 



C(l;fc~l)~7 



AA(0,1). (C.21) 



Combining this result with the convergence in distribution of the Uk show that there is a 
k{n) such that fifc is asymptotically normal. 
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